Interactive t-test: means, variability and p-values

A 6 mmHg (millimeters of mercury, a unit of blood pressure) difference between two treatments gives p ≈ 0.18 with 4 patients per group and p ≈ 0.014 with 12, with means and standard deviations (spreads) unchanged (constructed example). Here p is the p-value: the probability of a difference at least this large if both treatments had the same effect. You learn to compare a difference with its uncertainty, rather than read a p-value alone.

A course experiment whose values are computed within the stated model and are not research findings: The chapter: summary, formulas and FAQ (slides in French) · All experiments · Glossary: Student t-test.

z test: comparing a mean with a population value

When the population standard deviation σ is known, the mean of a sample is placed relative to the reference value μ₀ on the normal law. Change the observed mean X̄, sample size n or standard deviation σ and watch the orange line enter the red zone.

Show the formulas of the z test

$z = \frac{\bar{X} - μ_{0}}{σ / \sqrt{n}}$

$p_{two-sided} = 2 P (Z > | z |), p_{left} = P (Z < z)$

Where: X̄ is the mean observed on n patients, μ₀ the population reference value, σ its (known) standard deviation, z the resulting score and p the p-value; Z denotes a variable of the standard normal law N(0, 1).

Under H₀ (μ = μ₀), z follows the standard normal law N(0, 1); Z denotes a variable of this law. Two-sided (H₁: μ ≠ μ₀): H₀ is rejected if |z| > 1.96 at the 5% risk. One-sided left (H₁: μ < μ₀): rejected if z < −1.645. If σ is not known but estimated by the sample standard deviation s, the Student t test is used (t law with n − 1 df).

Reading the p-value. State a null hypothesis H₀ and an alternative H₁. Under H₀, the p-value p is the probability of a test statistic at least as extreme as the one observed. Set the threshold α in advance, often at 5%, then reject H₀ when p < α.

What does the α threshold control?

Under H₀ and the test assumptions, α limits the false rejection rate over repeated experiments. Interpret a significant result alongside the effect size and its uncertainty.

Two samples: when one point tips the test

Here σ is unknown: it is estimated from the data, and this is the Student t test. A single misplaced point can create or erase significance.

Show the formulas of the two-sample Student t test

$t = \frac{Δ}{\sqrt{\frac{s_{A}^{2}}{n_{A}} + \frac{s_{B}^{2}}{n_{B}}}}, Δ = {\bar{X}}_{B} - {\bar{X}}_{A}$

$ν = \frac{{(\frac{s_{A}^{2}}{n_{A}} + \frac{s_{B}^{2}}{n_{B}})}^{2}}{\frac{{(s_{A}^{2} / n_{A})}^{2}}{n_{A} - 1} + \frac{{(s_{B}^{2} / n_{B})}^{2}}{n_{B} - 1}}$

Where: X̄_A and X̄_B are the means of groups A and B, Δ their gap, s_A and s_B their standard deviations, n_A and n_B their sizes, t the test statistic and ν the number of degrees of freedom (Welch-Satterthwaite formula).

This is the Welch test, the course’s “heteroscedastic” case: the variances of the two groups are not assumed equal. It can be used systematically: it needs no prior Fisher F test on the variances, and it stays valid when they are equal. Under H₀ (equal means), t follows the Student law with ν df; H₀ is rejected if |t| exceeds the table t (shown in the lab).

Exercises

Taken from the course and its tutorials (no exam paper). Try before opening the answer.

Blood pressure (course). 36 treated patients have a mean arterial pressure of 92 mmHg; the reference value is 100 mmHg, with σ = 15 mmHg. Does the treatment change the pressure (two-sided, 5%)?
Show the answer
z = (92 − 100)/(15/√36) = −8/2.5 = −3.2. |z| = 3.2 > 1.96: H₀ is rejected. p = 0.0014, below the 5% threshold.
Reading scores (course). Three patients score 40, 35 and 30; the population has μ₀ = 50 and σ = 10. Which ones are significantly below (one-sided left, 5%)?
Show the answer
z = −1; −1.5; −2 and p = 0.159; 0.067; 0.023. Only the third patient (p < 0.05) is significantly below; the other two remain compatible with normal variability.
Cholesterol in men and women (course tutorial). Men: n = 34, mean 2.30 g/L, standard deviation 0.55. Women: n = 32, mean 2.18 g/L, standard deviation 0.54. Do the means differ?
Show the answer
Welch test (unpaired): t = 0.89 with ν ≈ 64 df, p ≈ 0.37 with these rounded summaries (0.357 reported in the course). p > 0.05: H₀ is not rejected, no difference is shown.
Broca’s area, left and right (course). 12 measurements on each side; means 36 and 70.25; variances 88.73 and 22.02. Do the Student t test by hand.
Show the answer
Welch test: t = (36 − 70.25)/√(88.73/12 + 22.02/12) = −11.27; ν = 16.1 df; table t ≈ 2.12. |t| > 2.12: H₀ rejected, p < 0.0001 (about 5 × 10⁻⁹). The course reads the table at 22 df (2.074): same conclusion. With two groups of equal size, t is the same with or without a pooled variance; only the df change.
The moving point. In the two-group lab, find the positions of the orange point that make the difference significant. Why does the result become non-significant again when the point moves further away?
Show the answer
Significant between 133.5 and 157 mmHg. Beyond, the point still widens the gap Δ between the means, but it inflates the standard deviation of group B, hence the standard error of Δ, even more: t decreases and p goes back above 0.05.

Try it and interpret

Keep the means and standard deviations, then vary the number of patients: the p-value changes while the difference stays the same. Make the two means equal: t (the difference in standard-error units) is 0 and p is 1.

Is a difference between means large relative to its uncertainty?

The test divides the difference between the means by its standard error. This error grows with variability and falls as sample size increases: the same difference can therefore stand out from sampling chance, or be lost in it. Significant does not mean important.

Show the formulas

$t = \frac{Δ}{\sqrt{\frac{s_{1}^{2}}{n_{1}} + \frac{s_{2}^{2}}{n_{2}}}}, Δ = {\bar{X}}_{1} - {\bar{X}}_{2}$

$ν = \frac{{(\frac{s_{1}^{2}}{n_{1}} + \frac{s_{2}^{2}}{n_{2}})}^{2}}{\frac{{(s_{1}^{2} / n_{1})}^{2}}{n_{1} - 1} + \frac{{(s_{2}^{2} / n_{2})}^{2}}{n_{2} - 1}}$

Where: X̄₁ and X̄₂ are the means of the two groups, Δ their gap, s₁ and s₂ their standard deviations, n₁ and n₂ their sizes, t the gap Δ in standard-error units and ν the number of degrees of freedom (Welch-Satterthwaite formula). Welch test: variances are not assumed equal, so no prior Fisher F test.

Definitions

Δ: The difference between the two means.
n₁, n₂: The sizes of the two groups, equal in this tool.
s₁, s₂: The standard deviation of each group, not assumed equal (Welch test). See the standard deviation in the normal distribution.
t: The difference Δ expressed in standard-error units.
ν: The degrees of freedom: here n₁ + n₂ − 2.
p-value: The probability, if the means are equal and the model is correct, of a difference at least as extreme. It is not the probability that the hypothesis is true. A small one flags a difference poorly compatible with equal means (H₀).

Assumptions and limitations

Explore the normality assumption. Jarque–Bera test.

Two-sided pooled-variance test for two independent groups of equal size: check normality and equal variances. Paired measurements need another test. Displayed points are constructed from the entered summaries.

Go further

The course chapter: summary with the formulas, conditions of the test and FAQ in English; slides in French
The normal distribution: standard deviation, area and thresholds
The chi-square test: compare counts rather than means
Correlation and R²: what significance does not say

Cite this page

Voyant, C. (2026). Interactive t-test: means, variability and p-values. www.cyrilvoyant.com. https://doi.org/10.5281/zenodo.24136645. https://www.cyrilvoyant.com/tools/student-t-test/. CC BY 4.0.

BibTeX entry, cited in a LaTeX document with \cite{voyant2026_studentttest} (package url or hyperref):

@misc{voyant2026_studentttest,
  author = {Voyant, Cyril},
  title = {{Interactive t-test: means, variability and p-values}},
  year = {2026},
  howpublished = {www.cyrilvoyant.com},
  doi = {10.5281/zenodo.24136645},
  url = {https://www.cyrilvoyant.com/tools/student-t-test/},
  note = {Accessed [date]}
}

Course archive (all versions) · Biostatistics course (10.5281/zenodo.23620235).